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We  propose  to  specialize  the  CCK  duality  theory^  which  associates 
as  dual  problems  minimization  of  an  arbitrary  convex  function  over 
on  arbitrary  convex  set  in  n-space  with  maximization  of  a  linear 
fltnctlon  in  non-negative  variables  of  a  generalized  finite  sequence 
space  subject  to  a  finite  system  of  linear  equations,  tc  derive 
Kuhn -Tucker  Theorem  extensions  in  situations  involving  (pirtiol) 
differentiability  of  objective  and  constraint  functions.  There  are 
several  ways  to  procure  such  generalizations  as,  for  example,  by 
means  of  non-dlfferentlable  pnalcgs  of  quasi-saddle  point  conditions 
or  in  terms  of  a  saddle  point  criterion  Itself.  Since  ve  are  Interested 
hare  in  exploring  extensions  which  involve  some  differentiability 
conditions,  ve  shell  proceed  via  the  former  course  especially  since 
these  conditions  themselves  are  analogs  of  first  order  conditions  of 
the  saddle  point  criterion? 

For  our  purposes  then,  let  f(u),  and  C(u)  •  (g^(u),  gg(u) , . . .  ^(u) ) 

be  defined  over  on  open  convex  set  K  in  R  .  We  shall  say  that 

n  H  \ 

f(u)  is  simple  piecewise  differentlably  convex  if  f(u)  •  max  (f'J/(u)), 

>1,2,...  ,N 

where  f'J'(u)  Is  continuously  differentiable  and  convex  over  K.  We 
shall  as  me  that  C(u)  is  continuously  differentiable  and  concave, 
but  the  extension  to  sl^le  plecevlse  concave  functions  vlll  become 
apparent  during  the  course  of  proof  for  functions  of  this  class. 


*  See  Chornee-Cooper-Kortarek  [4]  and  [5]. 

2 

See  Ruhn-Tucker  [7]  and  Arrow-Hurvlcz-Uzawa  [l]. 

^  See  Arrov-Rurvicz-Usava,  ibid.,  where  the  authors  show  that  in  the 
case  of  differentiability  the  quasi-saddle  point  condition  lilies 
the  saddle  point  condition. 
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Theoren  (Generalized  Quasi -Saddle  Point  Theorem  for  Simple 
Piecewise  Dlfferentlably  Convex  Functions) 

Let  f(u)  and  Cfu)  have  the  properties  defined  above  and  consider 

the  minimization  problem 

min  f(u) 

subject  to  G(u)  *  0. 

Assume  the  constraint  Bet  C  =  (u  I  G(u)  *  0)  has  an  interior  point. *  Then 
u*  lr.  C  is  an  optimal  solution  to  the  minimization  problem  if  and 
only  if  there  exists  positive  vectors 


1*  -  Oil1*  ,nl2)  )  and  \*  -  (k[  *  ,...,k *,“*  )  such  that 

the  following  properties  hold: 

(l)  -r  (6f(^»  |  )4J)  ♦  ’  (a*(1)  |  )kl‘>  .  o 

J-l  i-1  u 


(2)  L  ni^  -  1  and  (3)  G(u*)T  •  G,  and  G(u*)  »  02,  where 
J* J  J  =  (j/f(j)(u*)  *  f(u*)) 

Preliminary  Imit  on  Canonical  Closure  for  Differential  Systems 

By  introducing  support  systems  for  both  objective  and  constraint 
(Unctions,  we  obtain  the  following  equivalent  semi-infinite  problem  (i) 
with  semi-infinite  dual  (il),  which,  for  the  moment,  we  write  In  general 
form. 


(I) 

min  z 

t  -  uTQ  *  d  ,  a  c  A 

a  a 

*  c&,  1  c  X 


L  d  T) 

a  ** 

r  n 


(n) 

♦  i  cix1 


•  r  Vo  4  f  piKi 

a  i 


-  1 
-  0 


A  0. 


^This  type  of  constraint  qualification  has  strong  intuitive  Appeal  especially 
in  the  case  of  non-differentiability. 

However,  it  is  known  that  non-differentlable  analogs  to 
the  most  general  constraint  qualification  for  which  differentiable 
Lagranglan  techniques  are  valid  (see  [6])  involve  support  systems  Which 
are  thesuselves  Parkas -Minkowski  systems.  (See  [4]  and  [5]). 

2IVctatlonally  speaking,  df|u#  is  the  gradient  of  f  evaluated  at  u*.  We  use 
superscripts  to  correspond  to  functions  and  subscripts  to  correspond  to 
elsmants  in  the  index  set.  Thus,  df^;  denotes  the  gradient  of  fTj/  evalu¬ 
ated  at  the  point  a  c  A.  For  convenience,  "a  c  A”  may  be  Identified  with 
"UjjCA",  when  A  —  Rn» 


-  3" 


Recall  that  a  system  of  linear  inequalities  la  canonically  closed 
if  It  has  Interior  points  and  the  coefficient  set  is  compact. 

We  need  the  following  lenma. 

Leans  It  Suppose  that  the  system  is  canonically  closed  and  that  u# 

solves  (i),  i.e.,  the  minimum  s#  ■  f(u#)  is  attained.  Then  in  the 

dual  expression,  (il),  for  z#,  the  only  supports  vhich  arise  are  those 

paaslng  through  the  point  (z#,  u#),  i.e.,  the  only  support  planes  with 

T*  4  0  and  1*  i  0  are  those  for  which  z.  *  uZQ  ♦  d  and  u_P.  •  c. . 
•  i  ♦  *  q  a  *ii 

Proof:  By  the  extended  dual  theorem,  there  exist  q,  X  cuch  that 


«£dT|*  +  £  c.X*. 
•  a  a  i  i 


We 


t  show  that  if  »)*  >  0,  then  z_  «  u*Q  ♦  d  and  If  X?  >  0, 
a  •  *  a  a  l 


❖i  -  V 


then 


first.  z_  -  urQ  id,  for  all  a.  Hence 
*  •  a  a 


Z  d  »)*  *  E  t.n*  -  L  (uj&  )  -  r_  -  L  (uTq  )  T*. 

" "  ~  a  a  a  a 

a 


a  a  w  a  „ 
a  a  a 


Therefore. 


z#-  Ed 


X  ♦  *  v?  ‘  t#  ’  a  (u*V  n3  +  l  cix*  *  -  -  ' 

(A)  -  E  (uj^)  n*  ♦  z  c4X*  i  0 


See  [4]  and  [5].  Note  that  canonical  closure  is  a  sufficient  condition 
but  not  necassarv  for  the  validity  of  the  extended  dual  theorem  as 
pointed  out  in  [*]. 


On  the  other  hand,  by  dual  feasibility, 


”1  t- 1 VI*  ♦  [  pix!> 

T 

However,  since  u#P^  *  for  all 

(B>  0  -  I  *  E  ujp^  *  . 

a  l 

Therefore  combining  (a)  and  (B)  we 

E  u3q  n*  -  E  c.A*. 

O  *  aa  i  1  i 


-  uj  (0)  ■  0. 


i,  we  can  rewrite  this  as  follows: 


Two  conclusions  follow: 

(cx)  ^  CuJ^  ♦  1%,  where  E  ■  1,  t|  •  0,  and 

a  01 

r#  *  ♦  d^.  Hence  z#  -  u ♦  dft  for  every  a  with  »|*>  0. 

(C2)  E  ujp^f  -  £  ctA*  ->  E  (uj»t  -  c^AJ  -  0 
Hence  AJ  >  0  implies  uJp^  ■  c^. 


Proof  of  Thao 


With  respect  to  the  minimisation  problem  of  the  Theorem,  consider 
the  particular  seal-infinite  equivalent 


min  t 


u) 


subject  to  i  -  uTdf^}  *  f(j)(i^)  -  df(^  ,  j  •  1,2 


,,N 


■5* 


for  all  Or  c  A,  where  A  \b  some  index  set  in  F  (e.g.  the  convex 
constraint  set  C).  Since  C  has  interior  points,  it  follows  that, 
this  linear  inequality  system  also  does.  Form  a  canonical  normalization^, 
(i*e. ,  divide  each  inequality  by  a  positive  constant  to  make  the  cum 
of  the  absolute  values  of  the  coefficients  sum  to  l),  to  obtain  an 
equivalent  system  with  bounded  coefficients  and  interlority. 


subject  to 


min  z 

Mo’* 


(I) 


r(J)<V 


>  o 


»Vi> 


,(0 


(1) 


(1) 


'(J’> 


where  J  -  1,2, ...,m,  and  O  c  A. 

Nov  form  a  canonical  closure  by  possibly  enlarging  the  index  set 
to  A  3  A  and  adjoining  the  corresponding  limiting  inequalities  which 
are  of  the  fora;  4^1  -  uTQ^  1  d^ 


„*  p<*> 
u  r  a 


for  a  c  A  -  A. 


Let  (I)  denote  this  new  canonically  closed  equivalent  (which  differs 
from  (i)  by  only  these  possibly  adjoined  inequalities  and  also  has 
interior  points). 

Nov  if  u*  is  optimal  for  (I)  It  is  also  optimal  for  the 
canonically  closed  equivalent  (I)  and  lemma  1  applies.  However,  any 
of  the  possibly  newly  adjoined  Inequalities  which  are  actively  involved 
in  the  dual  are  positive  multiples  of  differential  Lyperplaoes  already 
in  the  system,  for  suppose  one  of  them  has  a  >  0,  say, 

d^  with  ci  «  A  -  A.  Then  by  lemma  l,  the  support 

plane  4^*  -  »  d^  contains  the  point  (u*,  f(u»))  l.e.. 


See  (5),  P  Ilk 


1 
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H(^f(u*)  -  *  u*TQ^  ♦  or  equivalently,  the  plane 

H^z  -  uTQ^  ♦  is  tangent  to  the  surface  z  »  f^(u)  at 

the  point  u«.  Since  f^(u)  is  continuously  differentiable,  and 

since  ^ 'qz  S  u  d'^'  over  C,  this  tangent  plane  is  ulnque 

up  to  a  constant  positive  multiple,  and  therefore  ve  do  not  need  to 
adjoin  these  additional  inequalities.  A  similar  argvaent  obviously 
holds  for  the  constraint  functions. 

We  now  present  the  semi -infinite  dual  (17)  and  derive  the 
conditions  of  the  theorem. 


■»*  e  I  [r(j,(^,)u(^)  -  a’  *  ij  ♦ 


<  l(a’ 


subject  to 


E 

.  a  1  a 


-e 


J  a 


♦  ee  •  o 

i  a 


and  t{,  7  A  0. 

By  the  dual  theorem  there  exists  a  dual  optimal  solution  ( rj*,  7*). 

By  lemma  1  rj*  has  non-zero  coordinates  corresponding  only  to  support 

planes  passing  through  the  optimum  (u*,  z_),  i.e.,  those  gradient 

(J)  - 

tangent  planes  at  this  point,  one  for  each  function  fw/.  This  also 
applies  to  X*  and  constraint  functions  g^,  and  therefore  ve  may  write 
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n*  •  (^^1^  »  niN;)  •*•<*  x«  *  •••*  n»uBf  upon 

id  JD^D 


setting  qj^ 


(J)-(J) 

**  % 


lor 


1, 


N  an 


for 


1. 


ve  obtain  the  following  dual  optimal  conditions: 


(1) 

-  z  df<J' 

i3)  ♦ 

E  *  0 

1 

and  (?) 

« 

J 

=  1 

where 

-X  ,<*> 

and  X<‘>.  0. 

1toe  equality  of  dual  functionals  yields, 


f(u*)  ■  z#  •  E  f^(u*)  -  E  u*Tdf*j)  qj^  ♦  £  u*T  xi° 

J  J  1 

♦  L  (-g(l)(u*)  X*1*  , 


-  E  f(j;(u*)  q*J)  -  E  g(l)(u*)  X^1*  . 


J  1 

Since  f^ J)(u«)  if(u*)  for  all  J  and  g^(u*)  >  0  for  all  1,  It 
therefore  follows  that,  (3)  Eg^^u*)  ■  0.  Furthermore ,  since 

f(u*)  -  max  (f^(u*)},  it  follows  that  qj^  «  0  whenever 
f  ' '(u*)  <  f(u*),  giving  condition  (2).  Thus,  the  three  conditions 
of  the  theorem  are  proved. 

On  the  other  hand,  given  positive  vectors  q*  and  X*  satisfying 

conditions  (l),  (2),  and  (3)  with  respect  to  u*,  then  since  /  0 

•  -  % 

and  *  0  In  the  canonically  closed  system  (T),  we  obtain  dual 

feasible  solutions  upon  setting  qj^y  »  q^V-A*^  and 
Furthermore,  the  dual  objective  '’alue  is  E  f"^(u*)  q^^,  and  condition 
(2)  implies  that  f(u*)  •  E  f^J\u*)q^'  giving  dual  equality  of 
objective  functions,  thereby  proving  that  (f(u*),  u*J  is  optimal. 


Our  generalization  of  the  quasi -saddle  point  version  of  the  Kuhn- 
Tucxer  Theorem  is  not  56  general  as  ve  Day  possibly  get,  but.  it  does 
Indicate  a  unified  approach  to  studv  these  equivalences  under  more 
general  circumstances.  In  fact,  we  are  already  obtaining  results  lor 
generalized  saddle-point  equivalence  theorems  for  arbitrary  convex 
functions  over  F^.  Ibis  is  the  subject  of  another  paper  and  will  be 
reported  on  elsewhere. 

Already  these  methods  have  rhowr.  that  the  crucial  property  of 
the  constraint  functions  is  the  Farltas-Minltowslti  property,  which  Is 
a  property  of  the  functions  themselves  expressed  in  tero6  of  finite 
positive  linear  combinations  of  their  "gradients. "  Geometric 
qualifications  are  sufficient  restrictions  on  the  constraining 
functions  to  permit  such  Farkas-Mlnhows’-J.  expressions.  In  general, 
however,  it  may  be  necessary  to  go  beyond  the  natural  gradient 
inequalities  provided  by  the  constraint  functions  to  obtain  strong 
duality  results. 

In  conclusion,  we  illustrate  this  now  by  constructing  a  canonically 
closed  equivalent  for  the  one-variable  Slater  example  by  adjoining  a 
r  rv  variable  to  the  gradient  inequality  system  following  the  methods 
of  our  regularization  procedures  for  semi-infinite  programs. 

Restating  the  Slater  example,  we  have: 

(I) 


min  x 

subject  to  -(l-x)a  >  0  with  unique  optimum 

x#  «  1.  Introducing  a  differential  system  of  supports  to  contain  the 
optimum,  we  obtain  the  equivalent  problem: 

(I) 


min 


subject  to  2(lc)  x  >  1-a*  for  0  <  a  <  2. 


Sea  Salter  [7],  [*»]  P  2l6,  and  [5],  P  119 
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Let  M  and  V  be  large  positive  ambers,  either  real  or  non- 
Ar  chine  dean,  £.e .  larger  than  any  real  number-^  and  construct  the 
following  seal- Infinite  dual  regularizations. 


min 

subject  to 


<V 

Mt  ♦  X 

t  ♦  2(l-a)  x  >  l-a2, 
x  >  -V 
-x  >  -V 


o  <  a  <  2 


mx  t  (l-a*)  >-a  -  vx*  -  vx~ 

subject  to  E  1  *  M 

a  a 

E  2(i-a)  \ +  -  a'  -  l 

a 

X'e  >  o. 


Observe  that  problem  ( 1^)  is  canonically  closed  and  that 
t  >  0  is  included  In  the  Inequality  system  and  corresponds  to  the 
Index  point  a  ■  1.  As  stated  above,  N  may  be  viewed  as  real  or 
non- Archimedean,  and  therefore  ve  shall  derive  dual  optimal 
solutions  for  (l_)  and 


In  a  manner  which  Is  valid  for  either 


We  know  that  (t,x)  -  (0,1)  Is  (lR)-feaslble  with  functional 
value  1.  Thus,  we  search  for  a  solution  (t#,  x#)  with  objective 
value  <  1,  if  It  exists,  and  therefore  ve  assirne  x#  <  1.  Ely  lesma  1, 
this  opt  law  Involves  only  support  planes  which  are  tangent  to  it  ar.d 
therefore  Involves  only  Its  own  gradient  Inequality  with  index  poln* 
a#  ■  x#.  But  this  Implies  t#  -  ( 1  -a# ) 3  yielding  (Z^ ^-objective  value 
M(l-0^)*  ♦  Of.  Applying  the  usual  differential  methods  for  finding  a 
minimum  to  this  function  yields  the  Thy lor  expansion. 


See  (3]  pp  756-7 
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M(l  -  <*#)2  ♦  a#  «  ♦  M  (a#  - 


±\a 


)2  for  0<a#<2, 


ad  equation  which  is  obviously  valid  for  arbitrary  N.  This  tells 

1 


us  to  take  a.  = 


OM  -  i  4| 

— — j —  to  obtain  minimum  objective  value  — jjjj- 


<  1. 


Furthermore,  the  point  (t#,  x#)  -  12  (Ip)"f#<48l*,le 

because  t  >  -(a  -  ")a  =  1  -  a2  -  2  (l-a)x#  for  0  <a  <  2,. 

which  Is  a  restatement  of  (i^)-feaslbllity.  But  taking  ■  M, 


the  dual  variable  associated  with  the  binding  constraint,  and 


solution 


Xa  -  0  for  a  /  a#  and  X*  ■  X*  »  0,  yields  a  dual  (lip)* 

with  equality  of  dual  objective  functions,  and  therefore  shows  that 
In  fact  the  two  solutions  form  dual  optimal  solutions  for  problems 
(ip)  -( lip )  whether  M  is  viewed  as  real  or  non -Archimedean . 

Observe  that  the  dual  solution,  X*,  is  aa  extreme  point  of  the 
associated  generalized  finite  sequence  space  ^  and  as  such  the  non- 
sero  coordinate  is  linear  and  homogeneous  In  In  particular, 

Xg  -  N.  Two  courses  of  action  with  respect  to  N  are  now  open  to 
us?  First,  if  M  is  real,  we  nay  let  M  •*  «  so  that  (t#,  x#)  ■*  (0,1), 
the  solution  to  the  Slater  problem,  with  corresponding  dual  variable 
characterised  by  X  ■*  m.  Second,  viewing  M  as  non-Archlmedean, 


we  obtain  dual  optimal  solutions  in  Hilbert's  field  with  caamon 
objective  value  1  -  which  In  the  extended  ordering  Is  larger 

than  any  real  ntmber  less  than  1,  but  itself  Is  less  than  1. 


See  [X]  p  211 

See  [2],  where  this  statement  was  first  proved  for  finite 
linear  programing  over  non-Archlmedean  ordered  fields. 
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